Support weight sharing in QNN GPU#2325
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
vjatoth-qti
force-pushed
the
dev/vjatoth-qti/qnn-gpu-weight-sharing
branch
from
0c0236a to
24f1374
Compare
|
changing to draft since there has been no update on this PR since feb |
unnim-qti
force-pushed
the
dev/vjatoth-qti/qnn-gpu-weight-sharing
branch
from
24f1374 to
b4fc7bb
Compare
unnim-qti
force-pushed
the
dev/vjatoth-qti/qnn-gpu-weight-sharing
branch
from
9b2ce4d to
e36e46b
Compare
unnim-qti
force-pushed
the
dev/vjatoth-qti/qnn-gpu-weight-sharing
branch
from
e36e46b to
8c99b76
Compare
Co-authored-by: qti-mattsinc <mattsinc@qti.qualcomm.com>
| # filename matches what genai_config.json references. | ||
| actual_output_dir = output_dir | ||
| model_file_name = "model" | ||
| model_file_name = Path(onnx_file_name).stem if has_additional_files and onnx_file_name else "model" |
There was a problem hiding this comment.
Have we tried using resave_model in update_llm_pipeline_genai_config_gpu analogously to how it's used in the NPU path, instead of changing this file?
There was a problem hiding this comment.
Hi Matthew,
Based on my understanding, the resave function stores files in the destination cache. In the QNN GPU workflow, when the context_iterator_models split is disabled, a single ONNX model is generated instead of a composite model, unlike the HTP workflow.
In this scenario, the _save_model function in cache.py saves the ONNX file to the destination output directory with the hardcoded name "model.onnx". However, this is inconsistent with the naming convention used in genai_config, where the model is named "model_ctx" after the ctx_gen pass.
To keep the naming consistent with genai_config, I updated cache.py so that the file is saved as "model_ctx" instead of model. This aligns the output produced with the expected GenAI configuration naming convention.
5 participants
Describe your changes
Checklist before requesting a review
lintrunner -a(Optional) Issue link